feat(frontend): add HuggingFace media output rendering in result panel#5675
Merged
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #5675 +/- ##
============================================
+ Coverage 70.64% 70.91% +0.27%
Complexity 3438 3438
============================================
Files 1145 1146 +1
Lines 45105 45187 +82
Branches 4975 4994 +19
============================================
+ Hits 31863 32044 +181
+ Misses 11589 11495 -94
+ Partials 1653 1648 -5
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 361 | 0.221 | 26,330/37,180/37,180 us | 🔴 -14.9% / 🔴 +156.7% |
| 🔴 | bs=100 sw=10 sl=64 | 753 | 0.46 | 131,619/164,511/164,511 us | 🔴 +13.4% / 🔴 +63.7% |
| ⚪ | bs=1000 sw=10 sl=64 | 893 | 0.545 | 1,109,234/1,191,384/1,191,384 us | ⚪ within ±5% / 🔴 +23.2% |
Baseline details
Latest main 0485d02 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 361 tuples/sec | 424 tuples/sec | 830.77 tuples/sec | -14.9% | -56.5% |
| bs=10 sw=10 sl=64 | MB/s | 0.221 MB/s | 0.259 MB/s | 0.507 MB/s | -14.7% | -56.4% |
| bs=10 sw=10 sl=64 | p50 | 26,330 us | 23,538 us | 11,905 us | +11.9% | +121.2% |
| bs=10 sw=10 sl=64 | p95 | 37,180 us | 35,396 us | 14,486 us | +5.0% | +156.7% |
| bs=10 sw=10 sl=64 | p99 | 37,180 us | 35,396 us | 19,430 us | +5.0% | +91.3% |
| bs=100 sw=10 sl=64 | throughput | 753 tuples/sec | 810 tuples/sec | 1,082 tuples/sec | -7.0% | -30.4% |
| bs=100 sw=10 sl=64 | MB/s | 0.46 MB/s | 0.495 MB/s | 0.66 MB/s | -7.1% | -30.3% |
| bs=100 sw=10 sl=64 | p50 | 131,619 us | 121,048 us | 93,412 us | +8.7% | +40.9% |
| bs=100 sw=10 sl=64 | p95 | 164,511 us | 145,119 us | 100,489 us | +13.4% | +63.7% |
| bs=100 sw=10 sl=64 | p99 | 164,511 us | 145,119 us | 111,365 us | +13.4% | +47.7% |
| bs=1000 sw=10 sl=64 | throughput | 893 tuples/sec | 889 tuples/sec | 1,104 tuples/sec | +0.4% | -19.1% |
| bs=1000 sw=10 sl=64 | MB/s | 0.545 MB/s | 0.543 MB/s | 0.674 MB/s | +0.4% | -19.1% |
| bs=1000 sw=10 sl=64 | p50 | 1,109,234 us | 1,122,428 us | 925,459 us | -1.2% | +19.9% |
| bs=1000 sw=10 sl=64 | p95 | 1,191,384 us | 1,174,390 us | 967,138 us | +1.4% | +23.2% |
| bs=1000 sw=10 sl=64 | p99 | 1,191,384 us | 1,174,390 us | 997,758 us | +1.4% | +19.4% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,553.59,200,128000,361,0.221,26330.28,37179.66,37179.66
1,100,10,64,20,2654.50,2000,1280000,753,0.460,131619.10,164511.02,164511.02
2,1000,10,64,20,22401.94,20000,12800000,893,0.545,1109234.33,1191384.18,1191384.18
juliethecao
force-pushed
the
hf/08-result-media
branch
from
June 16, 2026 22:30
1d05e38 to
339130b
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends Texera’s HuggingFace operator experience by (1) detecting/rendering media-like outputs in the workflow result UI and (2) adding/expanding HuggingFace property-editor support (task/model selection plus image/audio inputs). It also includes backend HuggingFace codegen additions and a small schema/output-column handling adjustment.
Changes:
- Add frontend media-type detection helpers (+ unit tests) and use them in the result table and row-detail modal.
- Add HuggingFace Formly field components (model selector, image upload, audio upload) and task preview/visibility logic in the property editor.
- Add backend HuggingFace task codegen objects (image/audio/media/QA) and adjust output-schema/result-column handling.
Reviewed changes
Copilot reviewed 33 out of 35 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| frontend/src/app/workspace/service/workflow-graph/model/mock-workflow-data.ts | Adds a mock HuggingFace operator predicate for frontend tests/mocks. |
| frontend/src/app/workspace/service/operator-metadata/mock-operator-metadata.data.ts | Adds a mock HuggingFace operator schema and registers it in the mock schema list. |
| frontend/src/app/workspace/component/result-panel/result-table-frame/result-table-frame.component.ts | Adds is*Cell helpers delegating to media-type util. |
| frontend/src/app/workspace/component/result-panel/result-table-frame/result-table-frame.component.spec.ts | Adds tests around media URL detection for result cells and util consistency. |
| frontend/src/app/workspace/component/result-panel/result-table-frame/result-table-frame.component.html | Updates table cell rendering to show media placeholders/icons based on detected type. |
| frontend/src/app/workspace/component/result-panel/result-panel-model.component.scss | Adds modal/row-detail styling for media display and copy actions. |
| frontend/src/app/workspace/component/result-panel/result-panel-modal.component.ts | Builds per-field row entries with media metadata + copy-to-clipboard + media proxy URL resolution. |
| frontend/src/app/workspace/component/result-panel/result-panel-modal.component.html | Renders video/audio/image inline in the modal with copy fallback for text. |
| frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.ts | Adds HuggingFace task preview samples, task-based field visibility/validators, and Formly mapping adjustments. |
| frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.spec.ts | Adds tests for HuggingFace task preview behavior (known/unknown/empty/non-HF). |
| frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.scss | Adds styling for the HuggingFace task preview card. |
| frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.html | Renders the HuggingFace task preview card above the Formly form. |
| frontend/src/app/workspace/component/hugging-face/hugging-face.component.ts | Introduces a HuggingFace model/task selector Formly field with caching, pagination, and search. |
| frontend/src/app/workspace/component/hugging-face/hugging-face.component.spec.ts | Adds basic unit tests for exported constants/cache invalidation. |
| frontend/src/app/workspace/component/hugging-face/hugging-face.component.scss | Adds styles for the HuggingFace model select UI. |
| frontend/src/app/workspace/component/hugging-face/hugging-face.component.html | Adds the HuggingFace task dropdown, search, model list, and pagination UI. |
| frontend/src/app/workspace/component/hugging-face-image-upload/hugging-face-image-upload.component.ts | Adds client-side image compression + data-URL storage for HF image inputs. |
| frontend/src/app/workspace/component/hugging-face-image-upload/hugging-face-image-upload.component.spec.ts | Adds unit tests for image upload component state/error paths. |
| frontend/src/app/workspace/component/hugging-face-image-upload/hugging-face-image-upload.component.scss | Styles the image upload and preview UI. |
| frontend/src/app/workspace/component/hugging-face-image-upload/hugging-face-image-upload.component.html | Adds template for selecting/previewing/clearing uploaded images. |
| frontend/src/app/workspace/component/hugging-face-audio-upload/hugging-face-audio-upload.component.ts | Adds backend-stream upload + preview for HF audio inputs. |
| frontend/src/app/workspace/component/hugging-face-audio-upload/hugging-face-audio-upload.component.spec.ts | Adds a minimal spec scaffolding for the audio upload component. |
| frontend/src/app/workspace/component/hugging-face-audio-upload/hugging-face-audio-upload.component.scss | Styles the audio upload and preview UI. |
| frontend/src/app/workspace/component/hugging-face-audio-upload/hugging-face-audio-upload.component.html | Adds template for selecting/uploading/previewing/clearing audio. |
| frontend/src/app/common/util/media-type.util.ts | Adds isImageUrl / isAudioUrl / isVideoUrl helpers. |
| frontend/src/app/common/util/media-type.util.spec.ts | Adds unit tests covering detection across prefixes/extensions/query strings/CDN URLs. |
| frontend/src/app/common/formly/formly-config.ts | Registers new HuggingFace Formly field types. |
| frontend/src/app/app.module.ts | Adds HuggingFace components to the app module declarations. |
| common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDesc.scala | Adjusts result-column fallback logic in codegen and output schema. |
| common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/QaRankingCodegen.scala | Adds codegen for QA/zero-shot/similarity/ranking task family. |
| common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/MediaGenCodegen.scala | Adds codegen for prompt-driven media generation tasks (text-to-image/video). |
| common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/ImageTaskCodegen.scala | Adds codegen for image task family (image-only and image+prompt). |
| common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/AudioTaskCodegen.scala | Adds codegen for audio task family (ASR/audio-classification/TTS). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
Automated Reviewer SuggestionsBased on the
|
juliethecao
force-pushed
the
hf/08-result-media
branch
from
June 22, 2026 18:21
b83d948 to
1eeba34
Compare
juliethecao
force-pushed
the
hf/08-result-media
branch
from
June 26, 2026 20:19
de3f53d to
fa6cfd0
Compare
xuang7
reviewed
Jun 28, 2026
xuang7
left a comment
xuang7
left a comment
Contributor
There was a problem hiding this comment.
Left one comment. Could you rebase onto the latest main and also increase the test coverage, since this change is relatively large?
…nent Register the huggingface formly field type and declare HuggingFaceComponent in AppModule. Provides a task dropdown, paginated model list with client-side search, and per-task field state preservation when switching tasks.
The rxjs/no-implicit-any-catch ESLint rule requires explicit type annotations on error callbacks in .subscribe() calls. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…mponent Satisfies the rxjs-angular/prefer-takeuntil lint rule. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…mponent Fix setInterval/setTimeout leaks by tracking timer IDs and clearing them in ngOnDestroy. Remove takeUntil(destroy$) from shared module-level task fetch to prevent cache poisoning when the initiating component is destroyed. Remove unused TASK_TAG_MAP and TASK_NAMES exports. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…FaceComponent inFlightByTag was written but never read, so multiple component instances mounting for the same uncached task would each fire a full backend request. Add an in-flight guard that polls for the existing request's completion, matching the pattern already used for task fetches. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… in HuggingFaceComponent
Read X-Texera-Truncated header via { observe: "response" } to detect
when the backend's model list is incomplete. Show a notice prompting
users to search. When truncated, search queries are sent to the backend
search endpoint with debounce; otherwise local filtering is used.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…r lint rule Add back destroy$ and takeUntil(this.destroy$) on all subscribe calls. For the shared task fetch, add finalize() to reset tasksFetchSubscription when takeUntil fires before next/error, preventing the stale-guard bug the reviewer originally flagged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…anup Mirror the defensive pattern from the tasks fetch so that when takeUntil(destroy$) cancels mid-flight, inFlightByTag is cleared and subsequent component instances can re-fetch instead of polling forever. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Elliot Lin <36275109+ELin2025@users.noreply.github.com>
- Fix model polling indefinite loop by detecting cancelled in-flight fetch - Fix CSS class mismatch: hf-tasks-error → hf-error to match SCSS - Remove unused imports in spec file Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Catch HTTP errors inside switchMap so search stream survives failures - Use empty string instead of null when clearing model selection Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 38 integration tests covering task loading, model loading, pagination, search, task selection, and model selection logic. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Register the huggingface-audio-upload formly field type and declare HuggingFaceAudioUploadComponent in AppModule. Handles server-side audio storage via the /huggingface/upload-audio endpoint with local preview. Co-Authored-By: Anish Shivamurthy <anish@uci.edu>
…nel components Add 55 new tests across 5 spec files to cover previously untested branches: - media-type.util: non-string type guard (3 partials) - result-panel-modal: prettyRowJson, copyText, trackByEntryKey, non-string row values, null-tuple guard, rowData constructor path - hugging-face-audio-upload: localPreviewUrl in previewSrc, stale upload response/error guards, ngOnDestroy blob URL revocation - hugging-face: cached task/model error paths, in-flight poll paths (inFlightByTag and tasksFetchSubscription branches), destroy cleanup - result-table-frame: template rendering for video, audio, image, and plain-text cells with real row data
Signed-off-by: Julie Cao <116243642+juliethecao@users.noreply.github.com>
Contributor
Author
|
@xuang7 Done. I have addressed the review comment, added test cases, and rebased/resolved conflicts. I will check back here again once the Codecov report gets officially updated to see if I need to make more test cases as needed. |
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… into hf/08-result-media
Signed-off-by: Julie Cao <116243642+juliethecao@users.noreply.github.com>
# Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.
…erator property edit frame
Contributor
|
Hi @Ma77Ball, could you review this PR when you get a chance? |
Contributor
|
@Ma77Ball Please do the review. |
# Conflicts: # frontend/src/app/workspace/component/hugging-face-audio-upload/hugging-face-audio-upload.component.spec.ts # frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.spec.ts # frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.ts # frontend/src/app/workspace/component/result-panel/result-table-frame/result-table-frame.component.spec.ts # frontend/src/assets/sample-video.mp4
…ton in result-panel-modal
Ma77Ball
suggested changes
Jul 14, 2026
Ma77Ball
left a comment
Ma77Ball
left a comment
Contributor
There was a problem hiding this comment.
After this small fix, the pr overall lgtm and I can test it to approve.
Co-authored-by: Matthew B. <mgball@uci.edu> Signed-off-by: Julie Cao <116243642+juliethecao@users.noreply.github.com>
xuang7
requested changes
Jul 16, 2026
xuang7
left a comment
xuang7
left a comment
Contributor
There was a problem hiding this comment.
Thanks for the PR. The media rendering changes look good overall. One blocking issue: this branch still includes stale changes that would revert parts of the current main branch. The PR description also needs to be updated.
This reverts commit a22b2f9.
…-bypassing fallback on proxy failure
xuang7
approved these changes
Jul 18, 2026
xuang7
left a comment
xuang7
left a comment
Contributor
There was a problem hiding this comment.
LGTM! Please update the description to remove the reference to the stacked PR, since it has been merged.
…of page-local index
Contributor
Author
|
PR description has been updated! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
Render image, audio, and video outputs from HuggingFace tasks inline in the workflow result panel instead of displaying raw data URLs as text.
New file —
media-type.util.ts:isImageUrl— detectsdata:image/data URLs and common image extensions (.png,.jpg,.jpeg,.gif,.webp)isAudioUrl— detectsdata:audio/data URLs and common audio extensions (.mp3,.wav,.ogg,.m4a,.flac)isVideoUrl— detectsdata:video/data URLs, common video extensions (.mp4,.webm,.ogg), and thefal.mediaCDN host used by fal.ai text-to-video outputsChanges to
result-table-frame.component.{ts,html}:isImageCell/isAudioCell/isVideoCellmethods that delegate to the detection helpers<img>,<audio controls>,<video controls>elements render inline in the row detail modalcellMediaTypes[rowIndex][columnIndex]), instead of re-running the detection helpers and getCell on every change-detection cycleChanges to
result-panel-modal.component.{ts,html}:rowEntrieswith per-field media metadata on modal openngOnDestroyto avoid leaking object URLs for the life of the pageChanges to
result-panel-model.component.scss:**Changes to
operator-property-edit-frame.component.ts:⋆validation.show = true) for the image/audio upload and prompt-column fields on HuggingFace operatorsNew file —
media-type.util.spec.ts:fal.mediaCDN URL and empty/plain-string inputsAny related issues, documentation, discussions?
How was this PR tested?
media-type.util.spec.ts(22 cases): data URL prefixes, file extensions, case-insensitivity, query strings, the fal.media CDN URL, cross-type rejection, and non-string input handlingresult-table-frame.component.spec.ts: media cell icon rendering — Play Video/Play Audio/View Image indicators and plain-text fallbackresult-panel-modal.component.spec.ts: row-entry construction from data URLs vs. remote URLs, blob-fetch success/failure, the SSRF-safe proxy-failure fallback, clipboard copy, and inline video/imagesrcbindingng testWas this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Sonnet 4.6 in compliance with ASF guidelines